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ABSTRACT 



This "toolbox 11 offers suggestions about how and when to 
create objective tests. Such tests are sometimes a quick way to find out how 
students are doing, and sometimes they help students focus on what they are 
doing in class or help teachers define the content that is worth knowing. The 
following suggestions are offered for developing objective tests: (1) use 

different types of questions for objective tests (f ill-in-the-blank, short 
answer, multiple choice, true-false, and matching-style) ; (2) be careful to 

write clear directions; (3) take tests yourself before giving them to 
students; (4) if you do make a mistake, give the students the marks; (5) make 
objective tests easy to grade; and (6) avoid discussing perceived differences 
or difficulties in class. Objective tests should be used sparingly and 
carefully, but they sometimes are a useful tool for teachers. (SLD) 
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“Objective tests" usually refer to paper-and-pencil tests containing closed questions, where 
very little deviation is permitted from the 'right' answer as determined by the test designer. 
These typically include multiple choice questions, short-answer, true-false, matching or 
labeling activities which students complete under timed conditions with or without access to 
resources to assist them. Because such tests may focus mostly on recall or limited application 
of information, many instructors prefer to use more authentic or holistic assessment activities. 
However, objective-style tests offer some benefits which will be discussed below. 

The word 'objective' is a bit misleading. Such tests are subjective because they reflect the test 
designer's (usually instructor's) view of the world. The instructor decides what is true and 
what is false. In making up questions the instructor chooses what knowledge is most 
important, what questions are most worth asking about that knowledge, how that knowledge 
will be represented (what terms and language will be used), and what interpretations are most 
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acceptable as answers. Objective tests certainly do not provide the 'best' evidence of learner's 
knowledge just because their results are quantitative. 

The purpose of this Toolbox is to offer some simple suggestions about when and how to create 
objective tests. Stated simply, if you are using objective tests, you might as well make good use 
of them. This includes making them easy to grade and as student-friendly as possible. Note 
that this Toolbox is not a primer on the theories of test construction -- that is well past our 
purposes. 



Why create tests? 

There may be situations in your teaching when it is appropriate to use comprehensive, paper 
and pencil, objective tests. But the use of such tests may seem to contradict the philosophies we 
have stated in this book. Here are reasons why some instructors find there are times when tests 
are a useful evaluation method, even for adult learners. 

First, tests sometimes are the best way to give teachers a quick idea of how students are doing. 
Second, as teachers work through the process of creating a test, they make conscious decisions 
about what content is worth knowing. When this happens, comprehensive tests serve as 
guides for designing teaching plans. Third, a comprehensive test helps some students focus on 
what they are doing in class. If students know that a comprehensive test waits for them 
sometime soon, most will attend to the activities of the classroom in a more rigorous manner. 
Finally, when students do well on a comprehensive test, they often feel a real sense of success 
and accomplishment. The key, for us, was working to make sure that those students who 
studied hard did well. For many adult students, tests represent the "currency" of an 
educational experience. Doing well on a homework assignment or a short essay is one thing; 
however, doing well on a comprehensive test is something quite different. Many learners have 
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told us how surprised and pleased they were that they were learning so much -- when they 
successfully completed tests. 

Many of our adult students remember experiences where they have failed tests. Naturally, 
some test fear remains. Where this is the case, comprehensive tests can help re-establish a 
"success cycle" for learners. If learners begin to have a little success -- if they do well on an 
objective test, they will find out it feels good. We believe this feeling of success can encourage 
learners to work just a little harder to have more success -- because they enjoyed feeling good. 

The design of the test should never seek to trick or punish learners for their lack of knowledge, 
but allow them to show how much they have remembered. And we want to make it clear that, 
with comprehensive tests comes the redundancy of review activities that help students 
remember the information they will need to successfully answer the test questions. We believe 
teachers should review and review (in the most enjoyable ways possible) the material they 
consider most important so that their students will be so ready for the test that anyone who 
works in class will do well. 

Question: Is it better to use an 'objective' (closed question) or a 'subjective' (open question) 
test? 

Answer: Each type of test has good and bad points. For teachers, making both tests tends to be 
an equal amount of work. The question is: when do you want to do the work? Objective tests 
take a lot of time up-front -- to create them, but they are relatively easy to grade. The good 
point is that you can save the test (or question pool if you have a good computer) from one 
year to the next and cut down on your work in following years. An essay test is easier to 
create, but may take hours to grade. And, it doesn't get easier to grade year after year. You 
expend the same effort grading a test in year two than you did in year one. 
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The ability and learning styles of students makes a difference as well. Some adult students 
have a difficult time putting their thoughts into words especially under the pressure and time 
limits of a test situation, but may find it easier — more straight forward -- responding to 
objective questions. Others become frustrated by the limited nature of objective test questions, 
which do not allow them to demonstrate the unique understandings they have constructed in 
a particular course. They may prefer the freedom of open questions. 

How do I design and use objective tests? 

Following are suggestions, not rules. They vary according to the learners' needs, the course 
content, and the test situation. Overall, remember that 'objective-style' test-making is a labor- 
intensive activity. Prepare to spend a lot of time to develop really effective questions. 

1. Use Different Tvoes of Ouestions for Objective Tests 

IT— T * 



Following are some descriptions and samples of five different types of questions: fill-in- 
the-blank, short answer, multiple choice, true-false, and matching-style questions. 



• Fill-in-the-Blank Questions 

Fill-in-the-Blank questions provide a sentence where students must know and fill-in the 
appropriate word that completes the sentence. Because choices of words are not provided, it is 
difficult for students to guess. Fill-in-the-Blank questions are also good because they allow 
students to answer the question within the context of a larger sentence. Adult students who 
have difficult times reading and spelling can have troubles with fill-in questions. One way to 
make the questions easier and provide spelling and answer clues is to provide a word pool 
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below the questions for students to choose from. This changes fill-in questions into matching 
questions. 

AVOID . . . 

-Avoid using sentences straight from the course reading material, or students will assume they 
need to memorize material. 

-Avoid ambiguous sentences that could be completed in several different ways: Leadership is a 
process of . (communication? control? organizing? leading?) 

DO . . . 

-Do ask students to supply important information, not incidental words. 

-Do place the blank near the end of the question, so the sentence is easier to read and make 
sense of. For example: 

In England in the late 1700s when people started to invent machinery to do their work, launching the 
Revolution . (Industrial ) 



• Short Answer Questions 

Short answer questions can vary from having students write definitions to having them justify 
or support points from an argument. They give students a structure for answering, but allow 
more freedom in these answers. Short answer questions are relatively easy to grade and to 
design. They require students to create complete responses. Thus they reduce guessing and 
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offer more reliable assessment of learner understanding. Certain types of short answer 
questions also can help judge a student's ability to reason or to synthesize information. 

AVOID . . . 

-Avoid using an open-ended question if you want to determine students' understanding of a 
particular concept. For example, the following question is a good open-ended question for an 
essay response, but it would be unfair to grade student responses against one or two pre- 
determined answers: 

(poorer) How has e-mail affected the way humans interact with each other? 



DO ... 

-Do be very clear and specific in your wording. Use phrases such as these: 

Define the following terms: 

a. peer assessment 

b. interdependent learning . 

Name one advantage and one disadvantage of the Walkman Personal Stereo System. Give an example 
for both the advantage and disadvantage. 
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-Try asking students to make a choice and then defend it by providing their reasons. For 
example: 

Choose the number of the statement you agree with most, then write TWO reasons supporting your 
choice: 



1 Kolb's Learning Style Inventory is a good career indicator for adults. 

2 Kolb's Learning Style Inventory helps identify introverts and extroverts 

3 Kolb's Learning Style Inventory can help learners understand how they process 

information. 

• Multiple Choice Questions 

Multiple choice questions usually provide a question or statement stem with a number of 
alternatives from which a student might choose. The questions might be very easy (a simple 
definition); more difficult (where more than one answer is correct and students have to choose 
an alternative that includes more than one choice— (i.e.. both a and c are correct); or very 
difficult (i.e.. where students must infer a great deal from the text). For example: 

A.(easier) In what year did World War II end? 

a. 1862 

b. 1945 

c. 1965 

d. 1939 
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B. (more difficult ) Which of the following helped the Japanese people gain a stronger feeling of 
nationalism? 

a. Japan's isolation from the rest of the world. 

b. the success of Japanese sports industries 

c. the need to sacrifice personal pleasure to industrialize 

d. a and c 

e. all of the above 

f. none of the above 

C. (most difficult) Which of the following changes in family relationships are brought about by 
television? (Below the question in the space provided, support any answers you choose.) 

a. Families will become more isolated. 

b. Families won't stay at home much. 

c. Families will become less friendly. 

d. Children will live with their families longer. 

e. Children will become more difficult for their parents to handle. 
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AVOID . . . 

-Avoid using an incomplete statement to start the question, because it can be difficult to read. 
For example: (poorer)77ze year World War II ended in was : a) 1862, b) 1945, c) 1965, (d) 1939. 

-Avoid using a negative stem because it can confuse students. For example: Which was not a 
cause of the 1839 rebellion? 

-Avoid using ambiguous qualifiers such as "maybe", "often" or "sometimes". 

-Avoid offering two or more responses which are equally correct. 

-Avoid embedding two questions in one. 

-Avoid asking questions about incidental information. 

DO . . . 

-Do make up as many responses as you can for each question that are equally plausible. 

-Do try to make each response similar in length and parallel in grammatical structure. 

-Do check questions carefully for clarity and effectiveness. Try them out on colleagues. Try 
covering all responses to see if the question stands alone and can be answered with a simple 
statement or word. 

-Do craft questions to test learners' ability to judge and infer. Some instructors give learners a 
piece of text to read or a graph to interpret. Then the multiple-choice questions ask learners to 
infer meanings or to solve problems using the information. Such questions may ask learners to 
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judge a best interpretation (but the answer should clearly be the ""best" choice, and the 
instructor needs to be prepared to defend this choice). For example: 

Read the poem "Clarendon" below. Then choose the BEST statement to summarize the central theme of 
the poem: 

a) Human beings change most profoundly when they are facing death. 

b) Loving relationships are more important than achievements. 

c) One never know who one's closest friends are. 

d) Men do not enjoy the same quality of relationships as women. 

• True or False 

In True or False questions, students are asked to read a statement and tell whether that 
statement is either true or false. If students know the information, distinguishing between a 
true and a false statement is usually easy. If the student does not know the information, the 
result is the flip of a coin. Often teachers make True or False questions more difficult by having 
students tell why a false statement is false. Otherwise guessing is encouraged. 

True or False questions are quick to grade. However, it is difficult to write a completely true or 
absolute statement. Some instructors create negative statements based on the course material, 
and some plant 'tricks' in the statement to force students to read questions carefully. For 
example: 

In 1853, Commodore Matthew Perry became the first person from the Western world to fly into the 
airport at Tokyo. 
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In this question, a careless student might see both Matthew Perry-1853 and miss the fact that 
he sailed into the harbor and not flew into the airport. 

AVOID . . . 

-Avoid using generalizations, because they are rarely completely true. 

-Avoid negative statements, which are confusing for some students. 

-Avoid using very long statements, or. statements which contain two ideas. 

DO ... 

-Do use statements which are completely true or completely false. 

-Do design questions which ask students to distinguish between fact and opinion. 

-Do consider presenting students with a graph, chart of statistics or other piece of text. Then 
ask them to rate several statements as true or false based on their interpretation of the graph, 
chart, or text. 

• Matching Questions 

Matching questions work well for asking questions about people, dates and events, and 
vocabulary terms and definitions. However, they only test students' recall of information, and 
can encourage rote memorization if over-used. The matching exercise also may be more a 
search-and-eliminate activity than a demonstration of understanding. Some adult learners may 
understand the relationship between two items, but be confused by the presentation of these 
two items in different columns, isolated from meaningful context. 
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AVOID . . . 



-Avoid having lengthy lists of terms (more than 20) as this can confuse the students. 

-Avoid asking students to draw lines between the items. This can be confusing, especially in 
longer lists. 

-Avoid giving away the last few matches as freebies. Add extra choices to the list of possible 
definitions on the right. 

DO... 

-Do put columns all on the same page. 

-Do order each line from left to right so that the space is first, the term is second, and the letter 
and definition to be matched is on the far right. It makes the questions much easier to grade 
with a key. 

- Do make sure that the words and choices are mixed up. 

-Do provide very clear instructions. For example: 

Match the phrase in Column II with the correct word in Column I. Write the LETTER for the matching 
phrase in Column II on the line beside each number of Column I. 

Column I Column II 

2. Dufay A. Developed the first system of written notation for music. 



2. Leonin 



B. Wrote the first known opera. 



13 



2. Be Careful to Write Clear Directions 

Directions are often taken for granted by teachers who are quite test-wise. But you should not 
assume that students are as test-wise as you are. For example, we have seen teachers write a 
question like "Should the federal government support dictatorships in Middle East?" fully 
intending that students address the question "Why?" But, the teacher did not add the question 
"Why?" to the end of the first question, and one student simply answered "No." 



At the same time, good directions help guide students time through a test. For example, note 
the difference between the following directions. 



Question 1: Why has Western influence been important to the growth of Japan? 

Question 2: Explain three reasons why Western influence has been important to the growth of 
Japan. 

Directions can encourage students to attend to certain points in the test. For example, "Read 
the following statements. If the statement is true, write true in the space to the left of the 
question number. If it is false, write false in the space to the left of the question number. If the 
statement is false, give a one-sentence reason why it is false. Be careful. There are some "trick 
questions" here. 
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3. Take tests yourself before giving them to students. 

No teacher should ever give an test without first taking it first. Better yet, ask another teacher 
friend to take it. Teachers often become "Store blind." They get so close to the test that they 
can't see the problems with it. It is much easier to correct a confusing question before the test 
than deal with the impact of the confusion later. Don't assume your first attempt at a test is 
perfect. Essay questions, especially, are prone to problems, and teachers should exhibit care. 

4. If you do make a mistake, give students the marks. 

We recall teachers who have created confusing questions -- especially matching questions -- 
where there can be more than one correct answer, depending on how the question is "read" by 
students. As you grade the test, you may see examples of question problems you did not 
anticipate. Sometimes you can see, very clearly, that students are reading a question in a 
correct way -- but a way you did not intend nor anticipate. 

If this happens, there is no reason to go back over and re-correct every test. Simply add a point 
to every test paper -- admitting the difficulty when you review the test and noting that each 
paper, whether the student read the question in the way you intended or not, received an 
additional point. Such a tactic saves time and gives students the feeling that you are being 
more than fair. 



5. Make objective tests easy to grade 

With just a little care, tests can be made much easier to grade. Before you give any test, 
imagine how you will grade it. (Where will you sit? What will you do? What do you want to 
guard against?) For short answer questions, decide exactly what you are looking for in the 
responses before you start to grade. 
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For example, in multiple choice, one way to guard against cheating is to ask students to circle 
the number /letter in the choices and put the number /letter in the space to the left. Consider 
having students use an answer sheet or card that is separate from the question pages. (Warn 
students to check their answers very carefully! It's easy to mistakenly skip a question on the 
answer card. 



6. Avoid discussing perceived differences or difficulties in class 



When you are grading many tests, you may make errors. Sometimes these errors are simple 
mistakes in addition. Sometimes you have become un-focused and may have given too few 
points for essay or short-answer questions. 



When we review a test with a group of students, we ask anyone who finds a "problem" to jot it 
down on a piece of paper which they give us at the end of the class. If the problem is simple 
addition, they note this. If they believe we have erred in ways that are not fair to their answers, 
they outline why they believe they should receive more points than they have. (We encourage 
students to take stands and support these stands. When they write notes about what they 
perceive to be errors, they are actually attending to program objectives.) We take these notes 
seriously and add points to the test if we believe they are deserved. 
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What are the benefits of using objective-style tests? 

1. Because questions can be marked quickly or scanned electronically, they are useful when 
evaluating a large group of learners, or when instructor's time is severely limited. 

2. Grading of individual students' responses is not biased by the instructor's opinions. There 
are few interpretive variations affecting the ways different instructors mark the same tests. 

3. Learner success does not depend on the learners' writing skills. Nor can particularly fluent 
writers 'fudge' their lack of information or understanding. 

4. Test results provide quantities of data that can be analysed to determine patterns of 
students' learning, level of difficulty of the content or questions, clarity of the questions, and 
follow-up instruction required in particular areas of content. 

5. Some learners report a satisfying feeling of real accomplishment and mastery when they 
successfully complete an objective test. 



What are some concerns with using objective tests? 

1. The question design is restrictive, forcing learners to fit their understandings into the test 
designer's way of describing a concept. The closed questions also do not allow learners to 
show their own understandings or to explain their rationales for choosing one answer over 
another. The single-choice correct answer does not reveal learner knowledge that may be close 
but not identical to the instructor-created response. 

2. Emphasis is on reading. Learners with poor reading skills are disadvantaged, especially 
under time limits. Objective tests may demonstrate more the students' inability to read the 
questions rather than their level of conceptual understanding. 
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3. Some learners may guess at answers, or choose the correct answer without understanding or 
being able to explain or apply them. 

4. Objective questions sometimes focus narrowly on learners' ability to memorize information, 
rather than their ability to apply, interpret, judge, and draw conclusions. 

5. Questions are time-consuming to produce. Because of this, instructors often re-use questions 
-- which may not always adequately reflect the learning or meet the needs of each new group 
of students. 

6. Adults often have difficulty remembering large amounts of information, or recalling and 
using discrete pieces of information that are isolated from a problem's context. Therefore 
adults may perform poorly on such tests even when they understand and apply the concepts 
quite well. 

Overall . . . 

There are times and places for using objective tests to assess adult learning. Some learners 
actually prefer them to other modes of assessment. However, good tests are difficult to 
develop and they are limited in what they reveal about learner understandings. Instructors 
also need to remember that for many adults tests of any kind may create anxiety and recall 
negative experiences of failure. Closed-question tests may be particularly frustrating for 
adults. They see different perspectives and apply knowledge according to context, and have a 
hard time choosing options according to rigid alternatives. Our recommendation is to use 
objective tests sparingly and very carefully. 
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